Analogue Synaptic Noise - Implications And Learning Improvements

We analyse the effects of analogue noise on the synaptic arithmetic during multilayer perceptron training by expanding the cost function to include noise-mediated penalty terms. Predictions are made in the light of these calculations which suggest that fault tolerance, generalisation ability and learning trajectory should be improved by such noise-injection. Extensive simulation experiments on two distinct classification problems substantiate the claims. The results appear to be perfectly general for all training schemes where weights are adjusted incrementally, and have wide-ranging implications for all applications, particularly those involving "inaccurate" analogue neural VLSI.